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Abstract 

Unmanned  Aerial  Vehieles  (UAVs)  have  beeome  an  important 
component  of  military  aviation  operations  and  skilled  UAV 
operators  are  a  valuable  part  of  this  component.  Currently  there  is  a 
need  for  improved  methods  of  facilitating  the  development  of 
mission  level  skills  among  operators,  including  target  identification 
and  maintenance  of  navigational  awareness.  Toward  this  aim,  we 
examined  the  extent  to  which  transient  neurophysiological  states 
could  be  used  as  an  index  of  engagement  within  a  visual  detection 
training  paradigm.  Participants  learned  to  distinguish  stationary 
indicators  of  directional  change  in  movement  for  target  tanks  located 
within  a  complex  vehicle  formation  background.  Fast  alpha  activity 
(10-13  Hz)  one  second  before  targets  were  presented  differed  as  a 
function  of  type  of  error  that  would  be  made  and  task  difficult. 
Prestimulus  alpha  shows  promise  as  a  candidate  metric  for  on-line 
monitoring  of  learner  engagement  and  workload. 

Introduction 


Electroencephalographic  (EEG)  recordings  have  been  used 
extensively  as  an  index  of  task  engagement  and  working  memory  load 
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(Berka  et  al,  2007;  Gevins,  Smith,  McEvoy,  &  Yu,  1997;  Kerick,  Hatfield, 

&  Allender,  2007).  For  example,  inereases  in  frontal  midline  theta  aetivity 
(5-7  Hz)  and  deereases  in  both  slow  (7.5-10  Hz)  and  fast  (10-13  Hz)  alpha 
activity  are  associated  with  current  working  memory  demands  in  both  spatial 
and  verbal  tasks  (Gevins  et  ah,  1997;  Smith,  McEvoy,  &  Gevins,  1999). 
Alpha  activity  is  also  affected  by  training  and  practice,  with  increased 
activity  associated  with  increasing  skill  level  on  a  given  task  (Smith  et  ah, 
1999). 

To  date,  consensus  has  yet  to  be  reached  regarding  the  best  approach 
for  examining  spectral  changes  in  EEG  recordings  (see  discussions  in 
Klimesch,  Freunberger,  Sauseng,  &  Gruber,  2008;  Makeig,  Debener,  Onton, 
&  Delorme,  2004).  While  overall  changes  in  alpha  and  theta  range  activity 
have  been  shown  to  change  with  task  difficulty,  others  have  argued  for 
examination  of  spectral  changes  associated  with  particular  working  memory 
processes  or  task  locked  to  particular  events. 

Though  macro  level  changes  in  EEG  activity  show  promise  for  a 
wide  variety  of  applications,  considerably  less  attention  has  been  given  to 
micro  level  changes  (Huang,  Jung,  Delorme,  &  Makeig,  2008;  Mazaheri, 
Nieuwenhuis,  van  Dijk,  &  Jensen,  2009).  Micro  level  changes  have 
traditionally  been  examined  with  event-related  potentials  (ERPs).  While 
important  in  many  applications,  ERPs  may  provide  an  index  of  a  relatively 
small  portion  of  on-going  neural  activity  (Huang  et  ah,  2008;  Klimesch  et  ah, 
2008).  Additionally,  ERP  extraction  techniques  require  the  averaging  of 
neuronal  responses  time  locked  to  a  number  of  discrete  stimuli  that  may  not 
be  present  in  many  real  world  operational  environments  (Huang  et  ah,  2008). 
For  these  reasons,  methods  of  examining  both  tonic  and  phasic  fluctuations 
in  neural  activity  suitable  for  operational  use  remain  a  goal  of  many 
neuroergonomics  investigations. 

Simultaneous  monitoring  of  spectral  changes  stemming  from  both 
relatively  long-term  or  tonic  changes  in  levels  of  engagement  as  well  as  more 
rapid  phasic  changes,  such  as  from  event  related  spectral  perturbations 
(ERSPs),  show  promise  for  operational  neuroergonomics.  For  example, 
Sauseng  et  al.  (2005)  observed  that  event  related  synchronization  (ERS)  of 
alpha  range  activity  distinguishes  between  retention  and  active  manipulation 
of  visuospatial  information  in  working  memory.  Huang  et  al.  (2008)  have 
observed  tonic  changes  in  alpha  bandwidth  activity  coupled  with  phasic 
changes  in  multiple  bandwidths  during  periods  of  high  visuomotor  tracking 
error. 

An  approach  with  particular  practical  significance  would  be  to  utilize 
micro-level  or  phasic  bandwidth  changes  to  predict  transient  states  when  an 
operator  might  be  less  engaged  in  a  particular  task  (i.e.,  overloaded  or 
distracted)  and  thus  be  more  likely  to  be  error  prone.  A  recent  approach  for 
examining  micro  level  spectral  changes  shows  promise  in  this  regard. 


Examination  of  prestimulus  alpha,  which  is  spectral  activity  in  the  alpha 
bandwidth  occurring  immediately  prior  to  the  onset  of  a  stimulus,  is  one  such 
approach.  Examination  of  prestimulus  alpha  shows  promise  as  a  means  of 
predicting  when  alertness  may  have  temporarily  decreased  to  a  point  where 
errors  are  more  probable  (Ergenoglu  et  ah,  2004;  Mazaheri  et  ah,  2009). 

Eor  example,  using  magnetoencephalographic  (MEG)  recordings, 
Mazaheri  and  colleagues  (2009)  demonstrated  that  elevated  occipital  alpha 
activity  prior  to  the  onset  of  a  visual  stimulus  predicted  whether  or  not 
participants  would  make  an  error  in  an  upcoming  trial.  Elsing  EEG, 
Ergenoglu  and  colleagues  (2004)  observed  significantly  elevated  alpha 
activity  in  a  1  second  prestimulus  period  when  participants  missed  near 
threshold  visual  stimuli  relative  to  when  they  were  detected.  The  aim  of  the 
current  investigation  was  to  examine  the  extent  to  which  prestimulus  alpha 
activity  might  be  used  to  predict  an  operators’  current  level  of  engagement 
and  thus  predict  errors  before  they  occur  in  a  challenging  visual  detection 
task. 

Methods 


Participants 


Twenty-two  participants  (18-28  years,  M  =  23.27,  SD  =  2.62)  with  self 
reported  normal  or  corrected  to  normal  vision  and  hearing  voluntarily 
participated  in  the  study  after  providing  informed  consent.  Participants  were 
further  screened  for  far  and  near  static  visual  acuity  using  the  Snellen  and 
Rosenbaum  eye  tests,  respectively.  The  majority  of  participants  had 
completed  at  least  some  college  classes.  Participants  currently  enrolled  in 
University  courses  received  partial  credit  toward  a  class.  Participants 
recruited  from  the  community  were  provided  with  a  small  amount  of 
financial  compensation. 


EEG  Recording  and  Analysis  Procedures 


A  Neuroscan  NuAmps  40  Channel  Amplifier  (with  Neuroscan  4.4  software) 
and  a  40  channel  Neuroscan  QuickCap  were  use  to  collect  EEG  data.  The 
EEG  signals  were  band-passed  filtered  at  1  to  70  Hz  and  sampled  at  500  Hz. 
The  EEGLAB  toolbox  (Delorme  &  Makeig,  2004)  in  conjunction  with 
MATEAB  v.2007b  (The  Math  Works,  Natick,  MA)  were  used  for  analysis  of 
the  EEG  recordings.  After  collection,  EEG  was  re -referenced  to  the  average 


of  the  left  and  right  mastoid  processes,  and  low-pass  filtered  at  30  Hz.  The  1 
second  of  EEG  preceding  each  behavioral  response  was  subset  from  the 
overall  recorded  EEG,  divided  according  to  whether  the  response  was  a  hit, 
miss,  false  alarm,  or  correct  rejection.  Any  1  second  pre-response  epoch  that 
contained  activity  exceeding  +-75  pV  on  the  ocular  channels  was  rejected 
due  to  ocular  artifact  contamination.  The  mean  log  spectrum  for  the  set  of 
remaining  epochs  of  each  type  was  calculated,  and  the  peak  dB  power  in 
each  of  three  frequency  bands  (theta:  4-7.5  Hz,  slow  alphal:  7.5-10  Hz,  fast 
fast  alpha2:  10-13  Hz)  was  identified  separately  for  EEG  preceding  each  type 
of  behavioral  response,  in  each  of  the  two  dilficulty  levels,  at  three  electrode 
sites  of  interest,  Ez,  Cz,  and  Pz. 


Figure  1:  Example  image  from  the  normal  difficulty  condition  (notice 
that  none  of  the  beige  tanks  are  moving  in  the  opposite  direction  as  the 
non-tank  military  vehicles.  The  target  is  circled  in  red  (notice  that  it  is 
a  green  tank,  moving  in  the  opposite  direction  as  the  non-tank  military 
vehicles. 

Experimental  task 


Participants  performed  two  difficulty  levels  (Easy  and  Hard)  of  a  visual 
search  task  that  simulated  the  role  of  a  ElAV  operator.  In  both  difficulty 
conditions  the  target  was  defined  as  a  green  tank  heading  in  the  opposite 
directions  of  all  other  non-tank  military  vehicles  (distracters).  In  the  Easy 
condition,  only  the  target  (if  present)  could  be  heading  in  the  opposite 
direction  of  all  other  non-tank  military  vehicles  (see  Eigure  1).  However,  in 
the  Hard  condition  there  were  also  other  (distractor)  beige  tanks  that  could  be 


heading  in  the  opposite  direction  of  all  other  non-tank  military  vehicles  (see 
Figure  2).  The  added  variability  of  these  distractor  tanks  made  the  task 
considerably  more  diflhcult,  as  confirmed  with  pilot  testing.  This  increased 
difficulty  was  intended  to  increase  mental  workload  while  participants 
performed  the  dilficult  visual  search  task. 


Figure  2:  Example  image  from  the  Hard  condition  (notice  that  one  of 
the  beige  tanks  is  moving  in  the  opposite  direction  as  the  non-tank 
military  vehicles.  The  target  is  circled  in  red  (notice  that  it  is  a  green 
tank,  moving  in  the  opposite  as  the  non-tank  military  vehicles. 


The  experimental  task  was  written  and  displayed  using  Microsoft  Visual 
Basic  6  software.  Each  condition  of  the  task  consisted  of  200  static  images 
displayed  on  a  19  inch  CRT  monitor  (Dell  M992)  for  750  ms  each.  The 
interstimulus  interval  was  1.8  s.  Images  were  generated  from  a  static  image 
consisting  of  a  background  desert-like  scene  obtained  from  Google  maps. 
Each  scene  contained  15  military  vehicles  (i.e.,  tanks,  jeeps,  and  other 
vehicles)  obtained  from  a  UAV  simulator.  The  position  of  the  vehicles  was 
randomly  changed  in  each  scene.  Of  the  15  military  vehicles,  a  green  tank 
(vehicle  of  interest- VI)  was  always  present. 

In  both  conditions,  a  random  variable  was  used  to  generate  a  global 
direction  on  a  360  degree  axis  for  all  vehicles  to  face  within  each  generated 
image.  In  the  Easy  condition  it  was  only  possible  for  the  green  tank  to  violate 
this  directional  display  (via  random  variable)  and  become  a  target.  However, 
in  the  Hard  condition  it  was  also  possible  for  the  beige  tanks  to  violate  this 
directional  display  (via  random  variable).  To  increase  the  dilficulty  of  the 
task  in  both  conditions,  an  additional  random  variable  allowed  for  each 


individual  vehicle  to  deviate  from  the  global  direction  by  30  degrees. 
However,  as  apparent  in  figures  1  and  2,  it  is  still  possible  to  perceive  the 
global  direction  in  which  all  non-tank  military  vehicles  are  heading.  Random 
variables  were  also  used  to  generate  the  color  of  the  non-tank  military 
vehicles  (green  or  beige)  as  well  as  their  location  on  the  screen. 


Procedure 


All  Individuals  were  first  tested  to  ensure  that  they  had  normal  vision  as 
assessed  via  the  Rosenbaum  and  Snellen  metrics.  They  were  then  fitted  with 
the  Neuroscan  QuickCap  and  it  was  aligned  on  the  head  in  accordance  with 
the  standard  10-20  system.  Standard  EEG  saline  gel  was  used  to  ensure  a 
good  connection  between  the  electrodes  and  the  scalp  and  all  impedances 
were  measured  to  be  below  5  k  ohms.  Electrocortical  activity  was  recorded 
from  15  electrode  sites,  including  midline  sites  Ez,  Cz  and  Pz,  as  previous 
research  evidenced  their  effectiveness  as  indicators  of  visual  working 
memory  and  mental  workload  (Gevins,  1997,  1999;  Ergenoglu  et  ah,  2004; 
Mazaheri  et  ah,  2009).  An  in-cap  ground  located  just  anterior  to  Cz  was  used 
and  all  electrodes  were  referenced  to  an  electrode  placed  on  the  left  mastoid. 
However,  EEG  data  from  an  electrode  attached  to  the  right  mastoid  (also 
referenced  to  the  left  mastoid  during  recording)  was  also  recorded  to  allow 
for  an  averaged  reference  of  the  two  mastoids  to  be  computed  offline  for  sites 
Ez,  Cz  and  Pz.  Electrooculogram  activity  was  also  recorded  with  two 
electrodes,  one  placed  above  and  below  the  left  eye,  in  order  to  detect  ocular 
artifacts. 

Participants  were  then  briefed  with  task  instructions  and  were  shown 
examples  of  the  experimental  task  and  were  provided  with  a  short  practice 
session,  hollowing  this,  individuals  completed  the  two  difficulty  levels  of 
the  task  in  a  counterbalanced  order  as  behavioral  and  EEG  data  were 
recorded.  The  behavioral  data  consisted  of  participants’  response  accuracy 
and  response  time  (RT).  Responses  were  categorized  within  a  signal 
detection  framework  of  Hits  (detecting  the  presence  of  a  orientation  change 
in  the  VI  where  there  was  one),  a  Miss  (failing  to  detect  the  orientation 
change  of  the  VI  when  there  was  one),  a  Ealse  alarm  (reporting  an  orientation 
change  of  the  VI  when  there  was  not  one)  and  a  Correct  Rejection  (not 
reporting  an  orientation  change  when  in  fact  there  was  not  one).  Participants 
indicated  their  response  by  clicking  the  mouse  when  they  believed  a  target 
was  present 


Results  and  Discussion 


Behavioral  Data 

The  number  of  hits,  misses,  false  alarms,  and  eorreet  rejeetions  were 
ealculated  for  each  participant  in  each  condition.  Next,  d’  and  P  scores  were 
calculated.  This  calculation  revealed  that  4  participants  had  d’  scores  of  less 
than  .6  in  the  Easy  condition.  These  participants  were  eliminated  from  all 
subsequent  data  analysis.  Analysis  of  the  behavioral  data  for  the  remaining 
18  participants  confirmed  our  difficulty  manipulations.  Examination  of  the 
proportion  of  hits  revealed  that  participants  made  significantly  fewer  hits  in 
the  hard  detection  condition  (M  =  .51,  SD  =  .16)  relative  to  the  easy 
detection  condition  (M  =  .74  SD  =  .13),  t(17)  =  1.63,p  <.001  .  Eikewise, 
false  alarms  (indicating  a  directional  orientation  difference  for  the  target  tank 
when  one  was  not  present)  occurred  significantly  more  often  in  the  hard 
detection  condition  (M  =  .27,  SD  =  .15),  relative  to  the  easy  detection 
condition  (M  =  .16,  SD  =  .08),  t(17)  =  -4.3, <.001.  Average  d’  scores  also 
differed  significantly  between  the  easy  and  difficult  conditions,  t(17)  =  7.16, 
p  <.001,  with  means  of  1.76  and  .66,  respectively.  Average  P  scores  were 
.82  and  .85,  respectively. 

Prestimulus  EEG  Analyses 

A  2  (task  difficulty-  easy  and  hard)  x  2  (target  presence-yes  or  no)  by 
2  (accuracy  -correct  or  incorrect)  repeated  measures  MANOVA  was 
implemented  to  examine  relative  power  in  the  theta,  slow  alpha,  and  fast 
alpha  bandwidths  for  the  one  second  period  proceeding  each  stimulus 
presentation.  Separate  MANOVAs  were  analyzed  for  each  electrode.  Due 
to  space  limitations  only  analysis  of  Pz  is  presented  here,  though  it  should  be 
noted  that  similar  patterns  were  observed  at  electrode  sites  Ez  and  Cz.  At  Pz, 
a  significant  multivariate  three-way  interaction  was  observed  between  task 
difficulty,  target  presence,  and  accuracy,  P(3,15)  =  5.\2,p=  .01,  partial 
p,  =  .5.  Elnivariate  analyses  revealed  that  both  slow  Alphal,  P(l,17)  =  4.6, 

=  .04,  partial  .21,  and  fast  Alpha2  P(l,17)  =  5.\3,p  =  .03,  partial 
p  =  .23,  contributed  to  the  significant  multivariate  effect.  The  three-way 
interaction  is  depicted  graphically  in  Eigure  3  for  slow  alpha  and  Eigure  4  for 
fast  alpha. 


Alphal  at  Pz 


— • —  Easy  Condition 
-------  Hard  Condition 


Figure  3:  Slow  Alphal  at  Pz  as  a  function  of  Target  Presence,  Accuracy,  and 
Difficulty  Level.  Error  bars  reflect  the  standard  error  of  the  mean. 


Figure  4:  Fast  Alpha2  at  Pz  as  a  function  of  Target 
Presence,  Accuracy,  and  Difficulty  Level.  Error  bars 
reflect  the  standard  error  of  the  mean. 


Prestimulus  alpha  activity,  and  particularly  fast  alpha  (10-13  Hz), 
differed  significantly  between  the  types  of  errors  made  in  the  Easy  and  Hard 
condition.  In  the  Hard  condition,  fast  alpha  activity  increased  in  the  one 
second  period  immediately  prior  to  a  miss,  relative  to  correct  detections  and 
also  relative  to  false  alarms.  A  reverse  pattern  was  observed  in  the  Easy 
condition.  In  the  Easy  condition,  fast  alpha  did  not  differ  between  correct 
and  incorrect  trials  when  the  target  was  present,  but  decreased  significantly 
for  false  alarms.  The  largest  differences  in  prestimulus  alpha  activity 
observed  for  both  slow  and  fast  alpha  occurred  between  Ealse  Alarms  in  the 
Easy  and  Hard  conditions.  Both  fast  and  slow  alpha  increased  for  Ealse 
Alarms  in  the  Hard  condition,  but  decreased  for  these  same  target  absent 
error  types  in  the  Easy  condition. 

POTENTIAL  APPLICATION 

The  current  results  demonstrate  potential  for  using  on-line  monitoring  of 
phasic  changes  in  alpha  bandwidth  activity  as  an  index  of  when  an  operator 
may  be  more  error  prone  or  when  a  learner  may  be  reaching  a  state  where  he 
or  she  is  less  likely  to  benefit  from  an  instructional  strategy.  The  prestimulus 
alpha  activity  examined  in  the  present  experiment  reflected  both  task 
difficulty  and  the  type  of  error  likely  to  be  made.  Eor  example,  if  alpha 
levels  increased  significantly  and  participants  were  making  a  significant 
number  of  Ealse  alarms,  the  present  results  suggest  that  there  would  be  a 
greater  than  average  chance  the  participant  found  that  task  particularly 
difficult.  However,  this  same  pattern  of  Ealse  alarm  errors  coupled  with 
decreased  alpha  activity  could  indicate  that  the  participant  had  become  less 
engaged  in  the  task  or  that  perhaps  the  task  was  not  challenging  enough. 
Observation  of  a  reverse  pattern  coupled  with  miss-type  errors  could  be  used 
to  confirm  this  interpretation  of  the  data. 

This  information  could  potentially  be  used  in  conjunction  with  other 
algorithms  to  improve  the  diagnostic  capabilities  of  an  adaptive  training 
paradigm.  On-line  monitoring  of  phasic  changes  in  alpha  bandwidth  activity 
coupled  with  performance  metrics  could  be  used  to  provide  an  indication  of 
when  a  pedagogical  change  was  needed.  If  alpha  activity  was  out  of  range 
(either  above  or  below  tonic  limits)  no  further  learning  would  be  expected  to 
occur.  Using  this  information  in  conjunction  with  the  pattern  of  behavioral 
performance  observed  could  be  used  to  distinguish  whether  to  make  the 
learning  environment  more  or  less  challenging.  Eurther  research  into  the 
applicability  of  these  results  for  determining  individual  differences  in 
learning  styles  and  for  use  in  a  neurophysiologically  based  adaptive  training 
program  are  currently  underway. 
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